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FINAL  REPORT  FOR  AWARD  #  W81XWH-05-1-0151 


GRANT  TITLE:  8-bit  vs  1 1-bit  Softcopy  Display  for  Radiology:  Diagnostic  Accuracy  and  Visual  Search 
Efficiency 

PRINCIPAL  INVESTIGATOR:  Dr.  Elizabeth  A.  Krupinski 

1.  Introduction 

Teleradiology  and  PACS  (Picture  Archiving  and  Communications  Systems)  have  changed  radiology 
significantly  over  the  past  20  years.  [1-6]  An  important  softcopy  display  issue  for  radiologists  is  monitor  bit 
depth.  Medical-grade  monitors  typically  display  8  bits  of  data.  This  is  sufficient  for  some  interpretation  tasks, 
but  many  images  are  acquired  at  higher  bit  depths.  This  results  in  a  potentially  significant  loss  of  information 
during  the  diagnostic  interpretation  process.  The  problem  that  we  investigated  in  this  project  was  whether 
diagnostic  performance  would  be  improved  with  an  1 1-bit  display  compared  to  an  8-bit  display  that  is 
comparable  in  every  other  way  (e.g.,  maximum  luminance).  The  major  hypothesis  of  this  proposal  was  that 
diagnostic  accuracy  would  be  higher  with  an  1 1-bit  softcopy  display  than  with  an  8-bit  softcopy  display  for  the 
detection  of  pulmonary  nodules  in  chest  DR  (direct  digital  radiography)  images.  The  second  major  hypothesis 
was  that  visual  search  efficiency,  measured  using  eye-tracking  methods,  would  be  greater  for  the  1 1-bit  than  for 
the  8-bit  display. 

2.  Body 

Softcopy  display  of  images  has  become  the  norm  rather  than  the  exception.  Even  though  softcopy  reading  has 
become  fairly  routine  for  many  radiologists,  there  are  still  a  surprising  number  of  advances  that  are  still  being 
made  with  softcopy  reading.  One  area  that  is  crucially  important  in  the  diagnostic  interpretation  chain  (from 
image  acquisition  through  display  and  diagnostic  viewing)  is  the  display  of  image  data  to  the  radiologist.  [7-8] 
There  are  still  many  questions  about  what  is  the  optimal  display  method  and  parameters  to  insure  the  best 
diagnostic  interpretation.  Some  of  the  basic  questions  include  required  spatial  resolution,  contrast  resolution, 
ambient  light  conditions  and  many  other  enviromnental  and  display  conditions.  Part  of  the  problem  with 
answering  many  of  these  questions  is  that  display  technologies  themselves  are  continually  changing,  and  with 
each  new  display  technology  the  questions  need  to  be  addressed  again. 

One  very  important  display  parameter  issue  facing  radiologists  today  is  bit  depth.  The  majority  of  both 
commercial  and  medical-grade  monitors  (cathode-ray  tube  (CRT)  and  liquid-crystal  display  (LCD)) 
manufactured  today  display  only  8  bits  (256  gray  levels)  of  data.  This  is  sufficient  for  some  radiographic 
interpretation  tasks  in  which  the  acquired  data  is  8-bits  or  less.  However,  many  radiographic  images  (e.g., 
computed  tomography,  direct  radiography,  full-field  digital  mammography)  are  acquired  at  higher  bit  rates 
(e.g.,  12-14  bits).  [9]  This  results  in  a  potentially  significant  loss  of  information  during  the  diagnostic 
interpretation  process  at  the  point  where  the  radiologist  views  the  image  data  on  a  softcopy  display  unit.  All  of 
the  acquired  gray  levels  are  not  displayed  at  once.  The  user  can  of  course  window  and  level  while  viewing  the 
image  and  thus  manipulate  the  displayed  gray  levels.  The  problem  is  that  all  gray  levels  are  not  visible  at  once. 
The  radiologist  takes  in  a  huge  amount  of  information  during  the  initial  view  of  the  image  (i.e.,  the  initial 
Gestalt  or  global  percept),  so  the  more  information  available  in  that  initial  Gestalt  view,  the  more  efficient  and 
more  informative  that  initial  impression  is  going  to  be.  [10-11]  Additionally,  even  though  window  and  level  are 
always  options  for  radiologist  to  view  more  of  the  image  data,  there  is  no  guarantee  that  every  radiologist  will 
use  these  tools  with  every  image.  More  raw-image  data  is  acquired  than  can  be  displayed  on  the  computer 
display  monitor. 

Compounding  the  problem  is  the  fact  that  the  human  visual  system  can  detect  at  least  1024  gray  levels,  and  that 
is  well  beyond  the  256  available  from  an  8-bit  display.  In  the  past  it  was  thought  that  people  could  see  only  a 
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limited  number  of  gray  levels,  but  that  was  with  lower  luminance  displays.  Increased  display  luminance  in 
recent  years  may  significantly  increase  the  number  of  visible  gray  levels,  making  it  much  more  important  to 
display  as  many  as  possible.  Softcopy  displays  capable  of  displaying  more  data  bits  (i.e.,  1 1  bits)  are  available 
and  can  provide  more  data  to  the  radiologist  in  one  single  presentation  (i.e.,  without  window/level 
manipulation),  potentially  improving  diagnostic  performance. 

Another  potential  benefit  is  that  providing  more  information  in  the  initial  image  display  may  reduce  the  need  for 
excessive  windowing  and  leveling,  reducing  the  time  needed  by  the  radiologist  to  render  a  diagnosis.  This  may 
reduce  fatigue,  an  issue  that  many  researchers  have  noted  to  be  a  potentially  significant  problem  in  the  digital 
reading  room  [12-17].  However,  with  improved  displays  overall  workflow  can  be  improved,  increasing  the 
number  of  images/cases  a  radiologist  views  in  a  day.  Fatigue  from  having  to  manipulate  the  image  data  can  also 
be  reduced  significantly  if  the  radiologist  has  the  best  possible  image  available  in  one  presentation.  The  trade¬ 
off,  of  course,  is  that  1 1-bit  displays  are  not  that  widely  used  and  thus  the  cost  now  tends  to  be  higher  than  8-bit 
displays. 

The  major  hypothesis  of  this  proposal  was  that  diagnostic  accuracy  would  be  higher  with  an  11-bit  softcopy 
display  than  with  an  8-bit  softcopy  display  for  the  detection  of  pulmonary  nodules  in  chest  DR  images.  The 
second  major  hypothesis  was  that  visual  search  efficiency,  measured  using  eye-tracking  methods,  would  be 
greater  for  the  1 1-bit  than  for  the  8-bit  display. 

3.  Key  Research  Accomplishments 

The  project  had  three  main  technical  objectives.  The  first  was  to  calibrate  three  sets  display  devices  from  three 
different  manufacturers  -  one  8-bit  and  one  1 1-bit.  We  obtained  sets  of  displays  and  video  boards  from  three 
companies  -  DataRay,  Dome/Planar  and  Totoku  and  determined  their  maximum  and  minimum  luminance 
values  after  calibrating  them  to  the  DICOM  standard.  All  monitors  were  calibrated  within  acceptable  range  of 
the  targeted  values. 

The  second  objective  was  to  carry  out  three  independent  Receiver  Operating  Characteristic  studies  using  the 
same  set  of  images  on  the  three  sets  of  monitors  at  each  of  three  institutions  -  the  University  of  Arizona,  the 
University  of  Maryland  Baltimore  and  the  University  of  Southern  California.  The  results  were  compared 
statistically  for  1)  differences  in  diagnostic  perfonnance  for  8-bit  vs  1 1-bit  monitors  overall,  and  2)  differences 
between  monitor  manufacturers  (DataRay,  Totoku,  Dome/Planar).  A  set  of  100  DR  chest  images  was  collected 
from  existing  databases  at  Arizona  and  Baltimore.  The  final  set  had  50  nodule-free  and  50  with  subtle 
pulmonary  nodules.  Nodule  status  was  verified  by  computed  tomography  (CT).  All  patient  identifiers  were 
removed  so  the  identity  of  the  patients  was  not  known  to  anyone  participating  in  the  study. 

Each  of  the  three  sites  recruited  six  radiologists  to  participate  as  observers.  Each  radiologist  viewed  all  100 
images  twice  -  once  on  the  8-bit  and  once  on  the  1 1-bit  monitor.  A  counterbalanced  presentation  design  was 
used.  The  task  of  the  observers  was  two-fold.  Their  initial  task  was  to  view  each  image  and  decide  whether  a 
nodule  was  present  or  absent.  They  then  provided  their  confidence  in  that  decision  using  a  6-point  scale  where  1 
=  nodule  absent,  definite  and  6  =  nodule  present,  definite.  They  indicated  nodule  location  if  one  was  present. 

The  confidence  data  were  analyzed  using  the  statistically  powerful  Multi-Reader  Multi-Case  ROC  technique 
(MRMC).  [18]  MRMC  computes  Az  for  each  condition  and  then  does  an  Analysis  of  Variance  (ANOVA)  on 
the  resulting  Az  outputs  to  determine  statistically  significant  differences  between  conditions.  The  data  were 
analyzed  overall  comparing  8-bit  with  1 1-bit  (combining  all  the  data  from  the  3  sites,  disregarding  monitor 
type);  and  separated  into  groups  and  analyzed  8-bit  vs  1 1-bit  for  each  display  manufacturer.  Whether  or  not  they 
used  the  window/level  processing  function  during  interpretation  of  each  case  was  recorded.  Total  viewing  time 
was  also  recorded. 
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The  third  objective  was  to  measure  visual  search  efficiency  as  a  function  of  monitor  bit-depth.  Visual  search 
efficiency  was  measured  on  a  sub-set  of  images  at  the  University  of  Arizona  (the  only  site  of  the  three  with  the 
eye-tracking  equipment)  and  the  results  compared  for  8-bit  vs  1 1-bit  softcopy  display.  From  the  set  of  100 
images  we  selected  a  sub-set  of  representative  images  (5  lesion- free  and  15  lesion-containing).  We  used  the 
4000SU  Eye-Tracker  with  head  tracker  (Applied  Science  Labs,  Bedford,  MA).  We  used  the  eye-position  data  to 
characterize  a  number  of  parameters  of  visual  search:  time  to  first  fixate  a  lesion,  total  search  time,  and  dwells 
associated  with  each  decision  type  (TP,  FN,  FP,  TN).  These  are  the  parameters  we  have  analyzed  in  a  number 
of  other  such  studies.  [19-27] 


4.  Reportable  Outcomes 


a.  Physical  Calibration 

Table  1  shows  the  final  maximum  and  minimum  luminance  values  for  the  three  sets  of  monitors  used  in  the 
study.  The  displays  were  calibrated  following  the  recommendations  of  the  DICOM  Standard  [28].  The  target 
luminances  were  500  cd/nr  for  the  maximum  and  0.6  cd/ni2  for  the  minimum.  All  monitors  were  within 
acceptable  range  of  these  targets. 


Table  1.  Maximum  and  minimum  luminance  values  for  the  three  sets  of  monitors  used  in  the  study 


Display 

11-bit 

11-bit 

8-bit 

8-bit 

Max  Luminance 
(cd/m2) 

Min  Luminance 
(cd/m2) 

Max  Luminance 
(cd/m2) 

Min  Luminance 
(cd/m2) 

Planar 

500.61 

1.00 

504.48 

0.84 

DataRay 

500.87 

0.68 

509.32 

0.68 

Totoku 

492.35 

0.60 

500.45 

0.63 

b.  Observer  Performance  Results 

1.  Overall  results  for  all  three  groups/sets  of  displays 

The  observer  confidence  data  pooled  from  all  three  displays  (Totoku,  DataRay,  Planar/Dome)  were  analyzed 
using  the  Multi-Reader  Multi-Case  Receiver  Operating  Characteristic  analysis  [18].  There  was  no  statistically 
significant  difference  in  area  under  the  curve  (Az)  performance  (F  =  0.0374,  p  =  0.8491)  as  a  function  of  bit- 
depth  of  the  display.  Average  Az  with  the  8-bit  display  was  0.8284  and  average  performance  with  the  1 1-bit 
display  was  0.8253.  The  mean  Az  values  for  each  display  are  shown  in  Figure  1.  Differences  between  the  three 
sites  overall  are  likely  due  to  differences  in  the  observers  (e.g.,  level  of  experience  of  the  readers),  although  it  is 
not  possible  to  rule  out  the  possibility  that  there  are  differences  between  the  monitors  themselves  that 
contributed  to  the  overall  differences  between  the  three  sites. 
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Figure  1.  ROC  Az  values  for  each  of  the  three  sets  of  8-bit  and  1 1-bit  monitors  -  Totoku,  DataRay, 
Planar/Dome. 

In  terms  of  viewing  time  (i.e.,  time  from  when  the  image  appeared  on  the  display  until  they  hit  the  next  button 
to  remove  it  having  rendered  a  diagnostic  decision),  the  data  were  analyzed  using  a  repeated  measures  Analysis 
of  Variance  (ANOVA).  There  was  a  statistically  significant  difference  in  viewing  time  as  a  function  of  monitor 
bit-depth  (F  =  10.209,  p  =  0.0014).  Although  the  difference  was  small,  viewing  times  were  significantly  shorter 
for  the  1 1-bit  than  the  8-bit  displays  overall.  There  was  no  difference  as  a  function  of  whether  or  not  a  nodule 
was  present  or  absent  (F  =  0. 182,  0.6697).  The  average  viewing  times  are  shown  in  Figure  2. 


□  Absent 
■  Present 


Figure  2.  Average  viewing  time  for  the  nodule  absent  and  nodule  present  cases  on  the  8-bit  and  1 1-bit  displays. 

The  final  parameter  evaluated  was  whether  or  not  the  readers  used  window/level  processing  during  the 
interpretation  process.  Overall,  there  was  no  statistically  significant  difference  in  the  percentage  of  cases  on 
which  window/level  was  used  (X  =  1 . 10,  p  <  0.05)  with  the  8-bit  versus  1 1-bit  display  (see  Figure  3). 
Preference  for  using  window/level  seemed  to  be  an  individual  trait  -  some  readers  used  window/level  a  lot  and 
some  readers  did  not,  but  an  individual  reader  used  it  about  the  same  with  both  displays. 
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Figure  3.  Percentage  of  times  window/level  was  used  or  not  used  with  the  8-bit  and  1 1-bit  displays. 

2.  Totoku  display  monitor  results 

There  was  no  statistically  significant  difference  in  area  under  the  curve  (Az)  performance  (F  =  0.1860,  p  = 
0.6843)  as  a  function  of  bit-depth  of  the  display.  Average  Az  with  the  8-bit  display  was  0.7474  and  average 
performance  with  the  1 1-bit  display  was  0.7307.  The  individual  Az  values  for  each  observer  are  shown  in 
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Figure  4.  It  can  be  seen  in  Figure  4  that  half  of  the  observers  performed  slightly  better  with  the  8-bit  and  half 
performed  slightly  better  with  the  1 1-bit  display.  None  of  the  individual  differences  were  significantly  different. 


□  8-bit 
■  11-bit 


Figure  4.  ROC  Az  values  for  each  of  the  six  readers  using  the  Totoku  8-bit  and  1 1-bit  monitors. 

In  terms  of  viewing  time  (i.e.,  time  from  when  the  image  appeared  on  the  display  until  they  hit  the  next  button 
to  remove  it  having  rendered  a  diagnostic  decision),  the  data  were  analyzed  using  a  repeated  measures  Analysis 
of  Variance  (ANOVA).  There  was  no  statistically  significant  difference  in  viewing  time  as  a  function  of 
monitor  bit-depth  (F  =  1.256,  p  =  0.2628),  both  for  nodule  present  and  nodule  absent  cases  (F  =  0.377,  0.5396). 
The  average  viewing  times  are  shown  in  Figure  5.  The  individual  viewing  times  for  the  8-bit  and  1 1-bit  Totoku 
displays  are  shown  in  Figure  6. 
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Figure  5.  Average  viewing  time  for  the  nodule  absent  and  nodule  present  cases  on  the  8-bit  and  1 1-bit  Totoku 
displays. 
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Figure  6.  Average  viewing  time  for  each  of  the  6  readers  using  the  Totoku  8-bit  and  1 1-bit  displays. 

The  final  parameter  evaluated  was  whether  or  not  the  readers  used  window/level  processing  during  the 
interpretation  process.  Overall,  there  was  no  statistically  significant  difference  in  the  percentage  of  cases  on 
which  window/level  was  used  (X  =  1 . 10,  p  <  0.05)  with  the  8-bit  versus  1 1-bit  display.  Preference  for  using 
window/level  seemed  to  be  an  individual  trait  -  some  readers  used  window/level  a  lot  and  some  readers  did  not, 
but  an  individual  reader  used  it  about  the  same  with  both  displays  (see  Figure  7). 


□  8-bit 
B11 -bit 


Figure  7.  Percent  of  cases  each  reader  used  the  window/level  (W/L)  function  on  with  the  Totoku  8-bit  and  11- 
bit  monitors. 


3.  DataRay  display  monitor  results 

There  was  no  statistically  significant  difference  in  area  under  the  curve  (Az)  performance  (F  =  0.4613,  p  = 
0.4986)  as  a  function  of  bit-depth  of  the  display.  Average  Az  with  the  8-bit  display  was  0.8168  and  average 
performance  with  the  1 1-bit  display  was  0.8326.  The  individual  Az  values  for  each  observer  are  shown  in 
Figure  8.  It  can  be  seen  in  Figure  8  that  five  of  the  observers  perfonned  slightly  better  with  the  1 1-bit  than  with 
the  8-bit  display.  None  of  the  individual  differences  were  significantly  different. 
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Figure  8.  ROC  Az  values  for  each  of  the  six  readers  using  the  DataRay  8-bit  and  1 1-bit  monitors. 

In  terms  of  viewing  time  (i.e.,  time  from  when  the  image  appeared  on  the  display  until  they  hit  the  next  button 
to  remove  it  having  rendered  a  diagnostic  decision),  the  data  were  analyzed  using  a  repeated  measures  Analysis 
of  Variance  (ANOVA).  There  was  a  statistically  significant  difference  in  viewing  time  as  a  function  of  monitor 
bit-depth  (F  =  9.656,  p  =  0.0020),  with  the  8-bit  viewing  taking  significantly  longer  than  the  1 1-bit.  There  was 
no  difference  as  a  function  of  whether  or  not  the  case  contained  a  nodule  (F  =  0.745,  0.3884).  The  average 
viewing  times  are  shown  in  Figure  9.  The  individual  viewing  times  for  the  8-bit  and  1 1-bit  DataRay  displays  are 
shown  in  Figure  10. 


□  Absent 
■  Present 


Figure  9.  Average  viewing  time  for  the  nodule  absent  and  nodule  present  cases  on  the  8-bit  and  1 1-bit  DataRay 
displays. 
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Figure  10.  Average  viewing  time  for  each  of  the  6  readers  using  the  DataRay  8-bit  and  1 1-bit  displays. 

The  final  parameter  evaluated  was  whether  or  not  the  readers  used  window/level  processing  during  the 
interpretation  process.  Overall,  there  was  no  statistically  significant  difference  in  the  percentage  of  cases  on 
which  window/level  was  used  (X  =  0.00,  p  <  0.05)  with  the  8-bit  versus  1 1-bit  display.  Preference  for  using 
window/level  seemed  to  be  an  individual  trait  -  some  readers  used  window/level  a  lot  and  some  readers  did  not, 
but  an  individual  reader  used  it  about  the  same  with  both  displays  (see  Figure  11). 


□  8-bit 
B11 -bit 


Figure  11.  Percent  of  cases  each  reader  used  the  window/level  (W/L)  function  on  with  the  DataRay  8-bit  and 
1 1-bit  monitors. 

4.  Planar/Dome  display  monitor  results 

There  was  no  statistically  significant  difference  in  area  under  the  curve  (Az)  performance  (F  =  0.0868,  p  = 
0.7802)  as  a  function  of  bit-depth  of  the  display.  Average  Az  with  the  8-bit  display  was  0.921 1  and  average 
performance  with  the  1 1-bit  display  was  0.9126.  The  individual  Az  values  for  each  observer  are  shown  in 
Figure  12.  It  can  be  seen  in  Figure  12  that  two  of  the  observers  performed  slightly  better  with  the  1 1-bit  than 
with  the  8-bit  display.  None  of  the  individual  differences  were  significantly  different. 
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Figure  12.  ROC  Az  values  for  each  of  the  six  readers  using  the  Planar/Dome  8-bit  and  1 1-bit  monitors. 

In  terms  of  viewing  time  (i.e.,  time  from  when  the  image  appeared  on  the  display  until  they  hit  the  next  button 
to  remove  it  having  rendered  a  diagnostic  decision),  the  data  were  analyzed  using  a  repeated  measures  Analysis 
of  Variance  (ANOVA).  There  was  no  statistically  significant  difference  in  viewing  time  as  a  function  of 
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monitor  bit-depth  (F  =  0. 1 1 1,  p  =  0.7393).  There  was  no  difference  as  a  function  of  whether  or  not  the  case 
contained  a  nodule  (F  =  0.024,  0.8768).  The  average  viewing  times  are  shown  in  Figure  13.  The  individual 
viewing  times  for  the  8-bit  and  1 1-bit  Planar/Dome  displays  are  shown  in  Figure  14. 


□  Absent 
■  Present 


Figure  13.  Average  viewing  time  for  the  nodule  absent  and  nodule  present  cases  on  the  8-bit  and  1 1-bit 
Planar/Dome  displays. 
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Figure  14.  Average  viewing  time  for  each  of  the  6  readers  using  the  Planar/Dome  8-bit  and  1 1-bit  displays. 

The  final  parameter  evaluated  was  whether  or  not  the  readers  used  window/level  processing  during  the 
interpretation  process.  Overall,  there  was  no  statistically  significant  difference  in  the  percentage  of  cases  on 
which  window/level  was  used  (X2  =  0.00,  p  <  0.05)  with  the  8-bit  versus  1 1-bit  display.  Preference  for  using 
window/level  seemed  to  be  an  individual  trait  -  some  readers  used  window/level  a  lot  and  some  readers  did  not, 
but  an  individual  reader  used  it  about  the  same  with  both  displays  (see  Figure  15). 
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Figure  15.  Percent  of  cases  each  reader  used  the  window/level  (W/L)  function  on  with  the  Planar/Dome  8-bit 
and  1 1-bit  monitors. 


c.  Visual  Search  Results 

Six  observers  at  the  University  of  Arizona  site  viewed  the  set  of  20  images  (15  with  nodules,  5  without)  on  the 
Totoku  displays  as  their  eye  position  was  recorded  [27].  Figure  1  shows  a  typical  search  pattern  of  one  of  the 
radiologists  in  the  study.  The  circles  represent  fixations  or  locations  where  the  eye  lands  with  foveal  (high 
resolution)  vision.  The  size  of  the  circle  represents  shorter  (smaller)  or  longer  (larger)  dwell  times.  The  lines 
connecting  the  circles  represent  saccades,  or  jumps,  between  the  fixations  and  indicate  the  order  in  which  the 
fixations  were  generated.  The  arrow  in  Figure  1  indicates  the  location  of  the  nodule.  In  this  case  the  radiologist 
fixated  the  nodule  and  reported  it  correctly  (a  true  positive  decision). 


Figure  1.  Typical  scanning  pattern  of  a  radiologist  searching  for  pulmonary  nodules.  The  arrow  indicates  the 
location  of  the  nodule. 

In  terms  of  total  viewing  time  (time  from  when  the  image  first  appears  until  the  observer  hits  the  “next”  button 
and  it  is  removed),  it  took  slightly  less  time  on  average  for  the  observers  to  view  the  images  with  the  1 1-bit 
display  (5.78  sec  vs  6.21  sec)  but  the  difference  was  not  statistically  significant  (t  =  1.009,  p  =  0.3151).  Figure  1 
shows  the  average  total  viewing  times  for  the  individual  observers  in  both  reading  conditions. 
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Figure  1.  Mean  total  viewing  time  for  the  individual  readers  using  1 1-bit  vs  8-bit  displays.  The  mean  is  shown 
as  well. 


The  next  eye-position  parameter  evaluated  was  time  to  first  hit  the  nodule  targets  with  foveal  (high  resolution) 
gaze  during  the  search  of  the  image.  The  results  for  the  true  positive  (TP  or  when  they  correctly  reported  the 
nodule  as  present)  and  false  negative  (FN  or  when  they  did  not  report  a  nodule  that  was  present)  decisions  are 
shown  in  Figure  2.  For  both  TP  and  FN  decisions,  time  to  first  hit  the  nodules  was  shorter  with  the  1 1-bit  than 
the  8-bit  display.  When  tested  with  an  Analysis  of  Variance  (ANOVA),  there  were,  however,  no  statistically 
significant  differences  as  a  function  of  bit-depth  (F  =  0.981,  p  =  0.3234)  or  decision  (F  =  0.002,  p  =  0.967). 
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Figure  2.  Average  time  to  first  hit  a  nodule  (msec)  for  true  positive  (TP)  and  false  negative  (FN)  decisions. 

The  third  eye-position  parameter  evaluated  was  cumulative  dwell  times  associated  with  each  decision  type. 
Briefly,  the  system  samples  the  eye  position  of  the  reader  every  1/60  second  and  assigns  an  x,y  coordinate  in 
image  space  (the  head  tracker  data  are  integrated  in  real-time  with  the  eye-position  data).  The  x,y  coordinates 
are  then  grouped  into  fixations  using  a  running  mean  distance  calculation.  Fixations  can  then  be  grouped  in 
clusters,  which  can  be  thought  of  as  a  circular  area  of  about  2.5  degrees  radius  with  its  center  at  the  mean  x,y 
location  of  the  group  of  fixations  contributing  to  the  cluster.  Cumulative  clusters  can  be  calculated  by 
combining  clusters  generated  when  a  reader  re-fixates  the  same  image  area  (at  any  point  during  search).  The 
location  of  the  clusters  can  be  used  to  associate  dwell  times  with  decisions.  True  positives  and  false  negatives 
are  defined  by  the  lesion  locations  and  any  cluster  that  overlaps  the  lesion  location  by  at  least  50%  is  considered 
a  "hit".  False-positive  locations  will  be  specified  by  the  readers  so  we  will  be  able  to  use  the  same  "hit"  criteria 
for  these  decisions.  True  negative  decisions  constitute  areas  that  are  lesion  free  and  receive  fixation  clusters. 
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The  average  cumulative  dwells  for  true  positive  (TP),  false  negative  (FN),  false  positive  (FP)  and  true  negative 
(TN)  decisions  are  shown  in  Figure  3.  It  can  be  seen  that  for  the  1 1-bit  display,  cumulative  dwell  times  for  each 
decision  category  were  lower  than  for  the  8-bit  display.  When  tested  with  t-tests  for  paired  observations,  the  TP 
(t  =  1.452,  p  =  0.1507),  FN  (t  =  0.050,  p  =  0.9609)  and  FP  (t  =  0.042,  p  =  0.9676)  were  not  statistically 
significant.  The  difference  for  the  TN  decisions  was  statistically  significant  (t  =  1.926,  p  =  0.05). 


□  8-bit 
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Decision 


Figure  3.  Mean  cumulative  dwell  times  (sec)  for  TP,  FN,  FP  and  TN  decisions. 

5.  Conclusions 

Overall,  there  was  no  statistically  significant  difference  in  observer  performance  (as  measured  by  ROC  Az)  as  a 
function  of  bit-depth  for  any  of  the  three  displays  devices  used  at  the  three  sites.  Some  observers  performed 
slightly  better  with  the  8-bit  and  some  with  1 1-bit,  but  even  on  an  individual  basis  there  were  no  differences  due 
to  display  bit-depth.  Performance  overall  varied  from  site  to  site  and  that  could  be  due  either  to  differences 
between  the  observers  at  the  three  sites  or  possibly  differences  between  the  three  sets  of  monitors  that  had 
nothing  to  do  with  bit-depth.  Although  all  three  sets  of  displays  were  set  to  approximately  the  same  maximum 
and  minimum  luminance  values  and  calibrated  to  the  DICOM  GSDF  Standard,  the  technology  used  in  each 
display  is  different  and  thus  could  lead  to  potential  differences  in  performance.  Further  investigation  of  some  of 
the  other  physical  characteristics  of  the  displays  could  be  done  in  the  future  to  determine  if  such  differences 
exist. 

There  was  a  statistically  significant  difference  in  overall  viewing  time  as  a  function  of  display  bit-depth, 
although  the  actual  time  difference  of  1-2  sec  is  relatively  small.  The  DataRay  displays  were  the  main 
contributor  to  this  effect  although  for  all  three  sites  there  was  a  slight  time  advantage  with  the  1 1-bit  display. 
Small  differences  in  viewing  time  may  contribute  to  improved  workflow  when  one  considers  the  very  large 
number  of  images  a  radiologist  views  each  day,  but  those  in  the  range  of  1-2  sec  observed  in  this  study  seem 
unlikely  to  affect  workflow  in  a  practical  sense. 

The  use  of  window/level  did  not  differ  significantly  as  a  function  of  display  bit-depth.  It  was  hypothesized  that 
since  the  8-bit  display  showed  fewer  gray  levels  compared  to  the  1 1-bit  display,  the  observers  would  have  to 
window/level  more  with  the  8-bit  display  to  visualize  all  the  gray  levels.  This  was  not  the  case.  Although  one 
could  see  the  effects  of  reduced  bit-depth  when  a  simple  geometric  pattern  (step-wedge)  was  shown  on  the 
displays,  when  a  complex  images  without  regular,  definable  edges  was  shown  it  was  generally  impossible  to  see 
any  differences.  The  use  of  window/level  seems  to  be  more  of  a  personal  preference  than  a  function  of  display 
bit-depth.  Some  radiologists  use  it  a  lot  and  some  rarely  use  it. 
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The  eye-position  study,  although  conducted  at  only  one  of  the  sites  (Totoku  displays)  revealed  some  interesting 
differences  in  visual  search  efficiency  that  correspond  to  the  observation  at  all  three  sites  that  total  viewing  time 
was  slightly  more  efficient  with  the  1 1-bit  displays.  Although  not  statistically  significant,  time  to  first  fixate 
(direct  the  axis  of  high  resolution  foveal  gaze)  on  the  nodule,  whether  reported  or  not,  was  shorter  with  the  11- 
bit  than  8-bit  display.  This  suggests  that  the  nodules  were  more  visible  and  thus  attracted  attention  earlier  in 
search  with  the  1 1-bit  display.  As  with  total  viewing  time,  however,  the  lack  of  statistical  significance  makes  it 
difficult  to  determine  if  these  relatively  small  differences  would  actually  impact  workflow.  The  time  to  render  a 
positive  decision  (true  or  false)  was  also  shorter  with  the  1 1-bit  display,  again  suggesting  that  it  may  have  been 
easier  to  discriminate  the  nodule  from  the  background  with  the  1 1-bit  display.  The  false  negatives  (missed 
nodules)  actually  attracted  more  viewing  time  with  the  1 1-bit  display  than  with  the  8-bit,  adding  credence  to  the 
idea  that  nodules  were  more  conspicuous  or  visible  with  the  1 1-bit  display.  The  observers  were  attracted  to 
these  more  visible  features  even  though  they  did  not  resolve  into  the  correct  perception  of  a  nodule  being 
present.  The  true  negative  decisions,  which  constitute  the  majority  of  clusters  generated  during  search  of  an 
image,  were  significantly  shorter  for  the  1 1-bit  display,  again  suggesting  a  potential  improvement  in  search 
efficiency  with  the  1 1-bit  display. 

Although  there  may  a  slight  advantage  with  an  1 1-bit  display  compared  to  an  8-bit  display  in  terms  of  overall 
viewing  time  and  visual  search  efficiency,  the  differences  are  relatively  small  and  in  a  practical  sense  do  not 
seem  likely  to  affect  daily  workflow  in  a  significant  manner.  The  general  lack  of  differences  in  diagnostic 
performance  for  any  of  the  three  displays  tested  suggest  even  more  strongly  that  an  1 1-bit  display  does  not 
provide  a  significant  advantage  over  an  8-bit  display.  Since  1 1-bit  displays  are  typically  more  expensive  than 
the  more  traditional  8-bit  displays  and  generally  require  a  special  video  board  at  an  additional  cost,  the  results  of 
the  present  study  indicate  strongly  that  there  is  no  practical  advantage  to  using  an  1 1-bit  display.  This  study  was 
limited  to  chest  images  with  subtle  pulmonary  nodules,  but  it  seems  unlikely  that  other  types  of  images  with 
other  lesions  are  likely  to  yield  significant  differences  in  diagnostic  or  search  behavior  as  a  function  of  monitor 
bit-depth.  Confirmation  of  this  would  certainly  require  future  studies. 
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